Estimating Correspondence in Digital Video

This paper addresses several estimation problems involving correspondence in digital video. We present three cases, in order of increasing complexity: affine transformations, projective transformations, and general correspondence. We review some recent research in the first two cases, and present a new framework for discussing the third case. Many image and video processing problems hinge on possessing a dense (subpixel-level) estimate of correspondence between a set of still images. Generally the cameras which produced the images are uncalibrated, i.e., their locations and orientations are unknown. The classical correspondence problem is a fundamental and difficult problem in computer vision, as evidenced by more than 30 years of research. Notable approaches include feature-based parameter estimation [18, 23], interpolation of feature correspondences [4, 11, 20], optical flow [1, 3], layered motion [2, 9, 22], and correspondence along conjugate epipolar lines [5, 6, 10, 12, 21]. While in some applications, an unstructured optical flow field may be an adequate representation of correspondence between an image pair, there are many practical situations in which a parameterized or structured correspondence is induced by the geometry of the cameras. In addition to coupling the correspondence to a physical modeling assumption, parameterized correspondence is generally more consistent, easier to manipulate, and gives more information about the relationships between images. Domains where parametric global motion models arise include orthographically projected images (6 affine parameters) or images taken by a rotating camera (8 projective parameters). In the first case, we compute the variance in estimation of affine parameters from noisy feature correspondences. We further discuss how multiple pairwise translation estimates can reduce the variance of the joint estimation of translation over an image sequence. In the second case, we demonstrate how the least-squares estimation